Organizational Approaches for Peer-to-Peer based Information Retrieval System

In recent years, Peer-to-Peer based information retrieval systems have received significant interest among computer scientists. A peer-to-peer based information retrieval system consists of a set of nodes connected in a peer-to-peer fashion. Each node hosts a document collection to share with other nodes and these nodes work collectively to provide information retrieval service to users. In such a system, an information retrieval task can be considered as a search session during which nodes forward the query to their neighbors, perform local searches and/or return search results. While the promise of this type of applications is attractive, the underlying technology is challenging. The lack of complete, up-to-date information of states of other nodes in the network requires sophisticated strategies for effective distributed search. In addition, the presence of concurrent search sessions adds an another level of complication: nodes may not be able to complete forwarding and perform local searches for all queries they have received in a timely fashion due to bandwidth and processing capacity limitations. This thesis frames a P2P IR problem into a multi-agent framework and attacks it from an organizational perspective by exploring various adaptive, self-organizing topological organizations and designing appropriate coordination strategies for large-scale agent organizations. Specifically, two protocols have been designed to create semantic based implicitly clustered agent organizations and explicit multi-level hierarchical agent organizations respectively in the context of single-query peer-to-peer based information retrieval systems, i.e, each external query is processed until completion before another external query is allowed to enter the system. In forming implicitly semantically-close clusters, agents exchange their resource descriptions to expand their local information about the content distribution over the network. Agents then prune the topology based on predefined rules. Two search strategies were evaluated on the reorganized topology. The experimental results demonstrated that the topology reorganization process combined with a context-aware search algorithm can improve considerably the information retrieval performance. In forming an explicit, multi-level topical hierarchical structure to facilitate locating relevant documents, agents join different groups in the hierarchy based largely on their content similarity. The group formation is achieved by organizing the agent-view structures properly so as to place semantically similar agents together to form explicit groups in an incremental and distributed manner. A context-aware search algorithm is also designed to take advantage of the hierarchical organization. During the search process, agents in the network follow various cooperation strategies to forward queries and return results in the network. …